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ABSTRACT 

We  consider  a  network  of  N  noniineariy  coupled  neuron-like  elements 
subject  to  Langevin  noise  and  a  weak  periodic  modulation.  Through  the  adi¬ 
abatic  elimination  proceedure,  the  dynamics  of  a  single  neuron  is  obtained 
from  the  coupled  stochastic  differential  equations  describing  the  network. 
The  bifurcation  properties  of  this  'reduced  neuron'  model  are  discussed, 
together  with  cooperative  stochastic  effects  (e.g.  stochastic  resonance)  that 
result  from  the  presence  of  the  modulation. 


Recently,  there  has  been  an  upsurge  of  interest  in  single  or  few-neuron  nonlinear 
dynamics  [1-4] .  However,  the  precise  relationship  between  the  many-ncuron  connected 
model  and  a  single  effective  neuron  dynamics  has  not  been  examined  in  detail.  Such  a 
reduced  neuron  dynamics  has  been  derived  recently  (bj  for  a  network  of  N  symmetrically 
interconnected  neuron-like  elements  embodied,  for  example  in  the  'connectionist"  models 
of  Hopfield  [6,7]  or  Shamma  [8j  (the  latter  corresponding  to  a  mammalian  auditory  net¬ 
work).  Through  an  adiabatic  elimination  procedure,  the  dynamics  (in  closed  form)  of  a  sin¬ 
gle  neuron  has  been  obtained  from  the  system  of  coupled  differential  equations  describing 
the  N-neuron  problem.  The  problem  has  been  treated  both  deterministically  and  stochasti¬ 
cally  (through  the  inclusion  of  additive  and  multiplicative  noise  terms).  It  is  important  to 
point  out  that  the  work  of  ref  [5]  does  not  include  a  priori  a  self-coupling  term;  rather,  that 
theory  results  in  an  explicit  form  of  the  self-coupling  term,  in  terms  of  the  parameters  of 
the  remaining  neurons  in  the  network  i.e.,  the  network  dynamics  is  reduced  to  a  closed 
equation  describing  a  single  neural  element  with  nonlinearity  determined  by  the  parameters 
describing  the  remaining  neurons  in  the  network.  This  'reduced'  or  'effective'  neuron 
model  b  expected  to  reproduce  some  of  the  gross  features  of  biological  neurons.  Multiplica¬ 
tive  noise  effects  in  the  model  have  been  discussed,  in  greater  detail,  in  [9],  The  fact  that 
simple  single  neuron  models,  such  as  the  model  to  be  considered  in  this  work,  can  indeed 
reproduce  several  features  observed  in  biological  experiments  has  been  demonstrated  [lOj 
through  the  construction  of  inter-spike-interval  histograms  (ISIHs)  using  a  Schmidt  trigger 
to  model  the  neuron.  The  results  of  this  simple  model  are  in  very  good  qualitative  agree¬ 
ment  with  data  obtained  in  two  different  experiments,  on  two  different  animals. 

In  this  work,  we  extend  the  work  of  [5j  to  include,  a  priori,  self-coupling  terms  (such 
terms  are  almost  always  present  in  electronic  neural  networks,  for  example)  as  well  as  a 
weak  low-frequency  periodic  modulation.  Thus,  we  consider  a  general  network  of  neuron¬ 
like  elements  described  by  the  dynamics, 
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c,  u,  V  tanh  u, 
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F,  ( t )  4  q  sinuU 
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Here,  u,  represents  the  membrane  potential  of  the  iA  neuron  with  input  capacitance 
and  a  leakage  current  due  to  the  tr ana- membrane  resistance  Rt.  F,{t)  is  taken  to  be  Gaus¬ 
sian  delta^correlated  noise  having  zero  mean  and  variance  of  (we  assume  that  the  noise 
sources  for  the  different  neurons  are  uncorrelated,  throughout  this  work)  and  the  dot 
denotes  time-differentiation.  It  is  important  to  note  that  an  equation  of  the  form  (1) 
describes,  in  general,  N  nonlinearly  coupled  bistable  oscillators.  This  model  is  also  the  con¬ 
tinuum  version  of  the  Hopfield  model  [7j  if  the  coupling  coefficients  Jl}  are  computed  using 
the  Hebb  rule.  We  do  not  assume  such  a  rule  for  computing  the  coefficients  J,t,  nor  do  we 
assume  that  the  matrix  J  is  symmetric.  Our  system  (1)  therefore  is  closer  to  neurophysio¬ 
logical  models  than  the  Hopfield  model.  We  now  assume  that  at  least  two  distinct  time¬ 
scales  exist  in  the  dynamics  described  by  (l).  Specifically  we  assume  that 


(2) 


This  is  tantamount  to  assuming  that  we  may  treat  the  neuron  described  by  the  membrane 
potential  Uj  as  a  'slow'  neuron  interacting  with  a  'bath'  of  N- 1  'fast'  neurons  that  are  in 
equilibrium.  We  may  then  adiabaticaliy  eliminate  the  bath  variables  from  (1)  and  write 
down  an  equation  for  the  variable  u,  in  terms  of  the  bath  variables.  The  proceedure  for 
doing  this  ha3  been  described  in  (5)  and  is  not  repeated  here.  Specifically,  an  N-body 
Fokker  Planck  equation  is  constructed  from  (I).  Using  Haken’s  slaving  principle  [11]  the 
N-body  probability  density  function  is  factored  into  a  density  function  p(u,)  describing  the 
'slow"  variable  u,  and  a  conditional  density  function  embodying  the  remaining  variables. 
This  permits  the  separation  of  the  Fokker  Planck  equation  into  fast  and  slow  components. 
The  Fokker  Planck  equation  for  the  bath  variables  is  integrated,  in  the  steady  state,  by 
invoking  a  local  equilibrium  assumption  (this  is  necessary  since  detailed  balance  is  usually 
not  satisfied  in  two  or  more  dimensions).  Ultimately,  we  arrive  at  a  Fokker  Planck  equation 
for  «,  whence  a  stochastic  differential  equation  for  the  variable  ut  (this  variable  now 
characterizes  the  'effective"  or  'reduced'  neuron)  may  be  written  down  by  inspection.  This 
equation  is, 

u ,  =  —  qu,  +  (3 tanhu ,  4  <5 sin  uit  4  \/o?  F(t )  , 
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In  carrying  out  the  proceedure  leading  to  (3)  we  have  assumed  further  that  the  modulation 
frequency  is  smaller  than  the  Kramers  frequency  of  the  unmodulated  system.  This  'adia¬ 
batic  assumption"  allows  U3  to  treat  the  modulation  as  being  approximately  constant  in  our 
solution  of  the  Fokker  Planck  equation.  This  assumption  is  also  a  cornerstone  of  the  adia¬ 
batic  theory  of  stochastic  resonance  [12]  on  which  our  subsequent  results  are  based. 
Further,  we  assume  that 


of  R,  «  2(7,  (t  ^  1)  . 


(5) 


This  assumption  (quantified  in  (5j)  guarantees  the  convergence  of  the  steepest  descent  tech¬ 
niques  used  to  evaluate  the  coefficient  fl  in  (3).  Note  also  the  absence  of  terms  involving 
coupling  between  pairs  of  "fast"  neurons;  these  terms  are  O  ( R,  (>,;  >  l)  or  higher  and 
can  be  neglected.  It  is  worth  pointing  out  that  the  adiabatic  elimination  proceedure,  based 
on  the  inequality  (2)  may  actually  have  some  basis  in  neurophysiology,  e  g.,  when  one  con¬ 
siders  the  interaction  of  a  single  neuron  with  a  'dendritic  bath’. 
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A  detailed  analysis  of  the  full  dynamics  described  by  (3)  is  beyond  the  scope  of  this 
Letter  and  will  be  presented  elsewhere.  In  the  following,  therefore,  we  focus  our  attention 
on  a  single  'slow'  neuron  u,  coupled  to  a  'fast'  neuron  u2  as  a  representative  example 
which  we  use  to  describe  the  cooperative  effects  that  arise  as  a  result  of  their  interaction. 
Since  neurons  are  (inherently)  bistable  elements,  it  is  instructive  to  study  the  bifurcation 
properties  of  this  system  which,  in  the  absence  of  the  noise  and  modulation  terms,  may  be 
described  by  the  potential  function, 


U(  “  i)  =  ~  u  ,2  -  0  In  cosh  u , . 


<C) 


For  positive  a  and  0,  the  potential  is  bimodal  if  0/a  >  1  with  the  minima  located  (for  large 

/3/a)  at  c  « ±  tanh  and  an  elliptic  point  at  u,-  0.  It  is  instructive  to  consider  the 
a  a 

effects  of  the  fast  neuron  dynamics  on  the  transition  to  bimodality  in  the  potential.  To  this 
end,  we  plot,  in  figure  one,  the  ratio  0 ja  as  a  function  of  the  resistance  R  7,  for  different 
values  of  the  variance  cr|  of  the  noise  source  in  the  fast  neuron.  In  this  figure,  we  take 
Jn—  1  and  we  have  chosen  a  to  guarantee  that,  in  the  absence  of  the  second  neuron 
(J12  =  0),  the  potential  corresponding  to  the  neuron  u,  Is  bimodal.  Since  the  ratio  0/a  also 
determines  the  barrier  height  (for  a  given  a),  one  observes  that  dramatically  different 
effects  are  obtained  depending  on  the  sign  of  the  product  J12J2l  in  (4).  These  effects  are 
also  readily  observable  in  equations  (4).  For  Jl7~  1  =  Ju,  the  coupling  to  the  'fast'  neuron 
is  seen  to  increase  the  ratio  0/a  upto  a  maximum  value,  occurring  at 
1  -  (1  -  2C2Jn/a  *}>/* 

R7  = - - — t - — ,  after  which  it  decreases  (note  that  the  quantity  inside  the 


'22 


square  root  must  be  positive  in  order  to  get  an  extremum  within  the  constraints  inherent  in 
the  inequality  (5));  the  opposite  effect  occurs  in  the  J7l  ---  - 1  case.  As  the  noise  a7 
increases,  the  deviation  from  the  Jl7  =  0  case  becomes  less  pronounced.  At  R 7  ~  0  and 
2(72/<t  2 ,  the  curves  for  J7l- 1  and  J2l=-l  intersect,  yielding  the  J  l7  0  result  (0/a  ■■■•-  J  ,,/f ,) . 
Beyond  this  intersection,  the  inequality  (5)  is  violated  and  we  do  not  expect  the  theory  to 
yield  accurate  results. 


We  now  consider  the  effects  of  the  deterministic  modulation.  Specifically,  we  are 
interested  in  the  phenomenon  of  stochastic  resonance  wherein  a  small  amount  of  noise  can 
introduce  correlated  switching  events  in  the  effective  potential  (6).  An  adiabatic  theory, 
valid  for  very  low  frequency  «  and  weak  amplitude  q  (such  that  there  is  no  switching  in  *v.e 
absence  of  noise)  has  been  developed  by  McNamara  and  Wiesenfeld  [12],  The  central  result 
of  this  theory  is  that  if  one  computes  the  signal-to-noise-ratio  (SNR)  of  a  nonlinear  .•>'  stem 
of  the  form  (3)  as  a  function  of  the  noise  variance,  then  the  SNR  passes  throng'  a  max¬ 
imum  at  a  noise  variance  approximately  equal  to  the  potential  barrier  height  Stochastic 
resonance  has  been  investigated  [13]  for  a  single  (isolated)  oscillator  of  the  form  (3),  with 
arbitrary  a  and  0  (see  also  the  recent  review  [14j).  In  the  current  context,  we  define  the 
deterministic  switching  threshold  as  the  critical  value  6,  of  the  scaled  modulation  amplitude 
6,  above  which  one  would  obtain  deterministic  switching  in  the  a 2  -  0  -  /  ?  case.  This  critical 
value  can  be  easily  found  to  be  given  by  5,  ~  j •>,+  /? tanh uf  where 

and  0^:  (0)rrj_o  Then,  in  order  to  satisfactorily  explain  the  sto¬ 
chastic  resonance  using  adfabatic  theory  we  must  ensure  that  ^<,\  and  w<wk,  the  Kra¬ 
mers  rate  for  the  unmodulated  system  (the  latter  conditio, i  is  also  necessary  in  the  deriva¬ 
tion  of  (3)).  Figure  two  shows  the  characteristic  signal-to-noise-ratio  (SNR)  curve 
(obtained  via  numerical  integration  of  (3))  as  a  fur  .Lion  of  oj.  The  curve  is  peaked  at 
a  J  —  f/0,  the  barrier  height  determined  from  (6).  The  effect  of  introducing  noise  of  varying 
amounts  in  the  fast  neuron  dynamics  is  also  elucidated  in  thus  figure  In  figure  three  we  plot 
the  peak  SNR  (computed  from  the  the  adiabatic  theory  [12,13]  for  c;  /  ’„),  normalized  to 
its  value  in  the  J ,2  0  case  (no  coupling  to  the  bath  neuron).  We  consider  the  I  J 2. 

case,  it  is  evident  that  the  efficiency  of  the  stochastic  resonance  process  is  enhanced  (for 
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>  1,  for  a  given  «.  Note  however,  that 


low  values  of  cri)  by  the  coupling  to  the  bat)',  neuron,  with  the  most  pronounced  improve¬ 
ment  occurring  for  low  noise  strengths  in  the  fast  neuron  Similar  effects  have  been 
observed  recently  in  a  mean  field  model  of  globally  coupled  bistable  oscillators  fl5j  The 
increase  in  the  SNR  with  R2  (keeping  /Z,  fixed)  is  seen  to  correlate  with  the  decrease  of  the 
ratio  0/a  (and  the  accompanying  decrease  in  the  potential  barier  height)  observed  in  fig¬ 
ure  one  for  J2 i~-~l  In  fact  one  may  show,  within  the  framework  of  the  adiabatic  theory 
[  1 2, 1 3j  and  for  large  P/a,  that  an  enhancement  of  the  peak  SNR  such  as  observed  in  figure 

£2  /  8 

2  (and  in  figure  3  below)  is  obtained  if  — -  ..  /  -f—  >  1,  for  a  given  «.  Note  however,  that 

g‘  V  “'ll 

increasing  o2  tends  to  decrease  P/a  as  well  as  the  effective  interaction  6.  This  explains  the 
degradation  of  the  effect  as  increases.  The  corresponding  effect  for  the  J :i  1  case  is  very 
small;  this  is  readily  understood  by  observing  that  the  potential  barrier  height  increases  for 
this  case,  as  R2  is  increased  from  zero.  Hence,  ar.  enhancement  m  the  SNR  (as  observed  in 
figure  2  for  the  J5l=-1  case)  does  not  occur  within  the  range  of  values  of  R2  that  is  allowed 
by  (2)  and  (5).  In  passing,  we  briefly  point  out  that  the  case  Jl2~  1  J2l  is  qualitatively 
similar  to  the  Jn~  1  =  -J2l  case,  insofar  as  the  magnitude  of  the  parameter  0  is  concerned; 
however,  the  effective  signal  amplitude  &  is  seen  to  be  lowered  in  this  case  (compared  to 
the  J,2=l  case)  so  that  the  signal-to-noise  ratio  is  also  degraded.  Similar  arguments  apply 
to  the  J  l2  =  —  1  =  J2t  case. 


The  analysis  of  the  preceeding  paragraph  underscores  the  importance  of  the  sign  of 
the  interaction.  This  is  also  obvious  from  the  definitions  (3)  wherein  we  observe  that  0  (and 
hence  the  potential  barrier  height)  depends  on  the  product  whereas  the  effective 

modulation  term  6  depends  only  on  Ju  (the  factor  G~l  also  plays  a  role  but  we  do  not  con¬ 
sider  its  effect  in  this  Letter).  Clearly,  in  order  to  achieve  the  best  possible  SNR  the  barrier 
height  should  be  decreased  and  the  effective  modulation  increased  (but  not  beyond  the 
threshold  for  deterministic  switching)  as  a  result  of  the  coupling  to  the  bath  neurons;  this  is 
best  achieved  by  having  (for  the  twoneuron  case  under  consideration  here)  Jl2>0,J2l  <0. 
In  neurophysiological  terms,  we  could  argue  that  having  a  distribution  of  excitory  and  inhi¬ 
bitory  interactions  provides  superior  performance  to  having  all  the  couplings  of  one  type 
(the  latter  situation  is  also  unlikely  to  occur  in  neurophysiology,  although  it  could  certainly 
be  realized  in  electronic  analogs). 


Finally,  we  consider  the  case  Jn  — 0.  For  this  case,  the  potential  in  the  absence  of  any 
coupling  (^n=0)  is  parabolic.  However,  as  apparent  in  figure  four,  the  coupling  to  the  fast 
neuron  can  render  the  effective  potential  (6)  bimodal  above  a  certain  critical  value  of  R2- 
.  .  C2  ]  1/2 

This  critical  value  is  given  by  R2c  = - -  ,  +  J-^y  -  2J  nJ2lR  i02  .  Unlike 

J\2Ji\R\0? 

the  preceeding  case,  however,  this  bimodality  (which  actually  corresponds  to  a  bifurcation 
in  the  most  probable  value  of  u,  (9,16))  only  occurs  for  Jl2J2l  >0  within  the  constraints 
imposed  by  (5).  Further,  large  noise  strengths  a2  tend  to  degrade  this  effect.  Nonetheless, 
it  is  apparent  that  one  will  observe  a  stochastic  resonance  effect  in  the  presence  of  the  cou¬ 
pling  to  the  fast  neurons  even  though  there  is  no  such  effect  in  the  isolated  (J,3=0)  case. 


The  above  work  may  be  extended  to  larger  networks.  Clearly,  as  evidenced  by  the 
structure  of  the  factor  G~l  in  (4)  as  well  as  the  nature  of  the  terms  occurring  within  the 
summations,  the  magnitudes  as  well  as  the  signs  of  the  coefficients  are  of  paramount 
importance  in  determining  how  the  network  performs.  Note  that,  for  a  general  network  of 
nonlinear  oscillators  the  coefficients  J,s  could  take  on  any  values,  as  long  as  the  system 
remains  stable  i.e.,  0>O  (if  Q<0  <  at,  the  potential  (6)  will  be  parabolic).  However,  to 
maintain  some  contact  with  neural  network  theory,  wc  have  assumed  |Jlf|<  1  m  all  our 
simulations.  The  calculations  reported  here  have  been  extended  to  larger  networks  of  the 
form  (l)  In  which  the  resistances  R,  and  the  variances  cr,‘(t  >  l)  are  drawn  from  a  uniform 
random  set  subject  to  the  condition  (2)  and  (5).  The  results  obtained  are  qualitatively  .sum 
lar  to  those  discussed  here  for  the  special  case  i  2,  and  will  Ik;  presented  in  an  upcoming 
publication  It.  is  tantalizing  to  speculate  that  neurophysiologic al  systems  constantly  adjust 
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the  synaptic  couplings  (which  may  depend,  in  general,  on  a  vast  array  of  internal 
parameters)  in  response  to  the  stimulus  and  noise,  such  that  the  potential  function  that 
characterizes  the  network  (for  our  model,  this  would  be  the  'reduced'  or  ’effective'  poten¬ 
tial  (6))  admits  of  more  than  one  minimum.  This  leads,  in  turn,  to  nonlinear  dynamic 
effects  such  as  stochastic  resonance,  in  which  the  noise  actually  enhances  the  flow  of  infor¬ 
mation  throught  the  system;  this  effect  appears  to  improve  as  a  result  of  the  connections  to 
the  'bath"  neurons.  As  pointed  out  earlier,  adjusting  the  internal  parameters  such  that 
0/a>  1  also  guarantees  stability;  in  the  neighbourhood  of  the  minima  of  (C>)  one  may  write 
[I7j  the  long-time  probability  density  function  for  u,  in  the  form  exp  [5]  where  5  is  a  gen- 

\  d*U  0  ° 

eralized  'entropy'  taking  the  form,  S  =  -  — - ^(cjifu,2.  S  is  a  Liapounov  function  in  the 

2  duf 

neighbourhood  of  the  potential  minima.  If  the  work  of  this  Letter  could  be  generalized  to 
the  point  of  solving  the  Fokker  Planck  equation  corresponding  to  the  fully  coupled  system 
(l)  in  the  steady  state,  the  potential  function  would  be  an  N-dimensional  hypersurface.  To 
date  there  does  not  exist  a  technique  for  deriving  such  a  potential  for  an  N-dimcnsional 
nonlinear  stochastic  system. 


From  the  standpoint  of  this  Letter  it  is  also  important  to  reiterate  the  beneficial 
effects  of  coupling  the  neuron  u,  to  the  'fast'  neuron  ti2.  The  coupling  dearly  enhances  the 
information  flow  (measured  by  the  SNR)  to  the  output  of  the  system  and  can,  for  not  too 
large  noise  variances  of  provide  signal  processing  capabilities  that  arc  outside  the  capabili¬ 
ties  of  the  original  oscillator  (J1;=0).  The  addition  of  noise  to  the  deterministic  dynamics 
significantly  enhances  the  signal  processing  characteristics  of  the  network.  This  was  pointed 
out  earlier  by  Buhmann  and  Schulten  [18,19]  who  stated  that  noise... is  an  essential  feature 
of  the  information  processing  capabilities  of  the  neural  network  and  not  a  mere  source  of 
disturbance  better  suppressed...' 
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Figure  Captions 

1.  Effective  nonlinearity  parameter  0/a  (computed  from  (4)  for  the  two-neuron  case)  vs 
R2 ■  [RiiJn,Jzz)  =  (10.0, 1.0, 1.0).  Solid  curves  represent  J l2  =  1.0  =  J2l  with  <71  =  0,10.0,20.0 
reading  from  the  top  curve  downward.  Dotted  curves  correspond  to  Jiub  12  =  1.0=  -  J2i 
with  =0, 10.0, 20.0  reading  from  the  bottom  curve  upward. 

2.  Signal-to-noise  ratio  vs  slow  neuron  noise  variance  a/.  Bottom  curve:  J  l2~  0  (isolated 
case);  top  curve:  J12=1.0=  -  J21,  fl2  =  0.1  ,o|  =  0;  middle  curve:  J,2=1.0  =  -  J7ll 
R3  =  0.1,ctj  =  5.0.  g=0.1  =  «;  frequency  resolution  =0.001.  Other  parameters  as  in  fig.  1. 

3.  Peak  signal-to-noise-ratio  vs.  R2  for  J12=1.0=- J21  and  <72  =  0 , 5.0 , 10.0 , 20,0  reading  from 
the  top  curve  doownward.  Other  parameters  as  in  fig.  2. 

4.  Effective  nonlinearity  parameter  0/a  vs  R  2.  (fZ,,  Jn  ,J-n)  =  (10.0,0.0, 1.0).  Solid  curves 
represent  J12— 1.0  —  J2l  with  <r2  =  0 , 5.0 , 10.0  reading  from  the  top  curve  downward.  Dotted 
curves  correspond  to  Jsub  12  =  1.0  “  ~  ^21  with  a2  —  0,5.0, 10.0  reading  from  the  bottom  curve 
upward. 
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